Machine Learning is an important area of Artificial Intelligence, A major area of Machine Learning involves the use of data to train a neural network, and algorithms for training neural networks have found numerous highly successful practical applications, e.g. in visual recognition, speech recognition, natural language processing and text mining. Algorithms for training neural networks are run either on general-purpose computers or on specialist computing hardware, such as Graphics Processor Units. Neuromorphic computers are a recently developed class of specialist computers with the following properties: (a) they use multiple computing units (cores) connected by a communication network, (b) there is no global memory storage and the only available memory is distributed between, and local, to the cores or to a node containing a group of cores; (c) communication between cores is carried out by the transfer of a special type of message, known as a “spike”, a message which indicates the occurrence of an event in the sending core; (d) processing on a core is activated asynchronously in an interrupt-driven manner, triggered principally by the receipt of a spike message at the core. Such a neuromorphic computer is exemplified by the SpiNNaker machine [1].
The invention described herein solves a long-standing problem of training a neural network on such a neuromorphic computer. The specific type of neural network addressed in the preferred embodiment of the invention is known as a Restricted Boltzmann Machine (RBM). The standard algorithm for training an RBM on a general-purpose computer is the Contrastive Divergence (CD) algorithm [2]. However, this algorithm is unsuitable in its standard form for training an RBM on a neuromorphic computer, as it requires the transfer of real-valued learnt parameter values between cores, or their storage in and retrieval from global memory, i.e. which is accessible to all the cores.
By employing the invention described herein, there is no requirement for the communication of parameter values between neurons, or for the accumulation and retrieval of stored changes to these parameter values using shared memory, thus providing a training algorithm which can be implemented efficiently on a neuromorphic computer system.